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Abstract 

In this paper, two tests, based on CUSUM of the residuals and least squares estimation, are studied to detect 
in real time a change-point in a nonlinear model. A first test statistic is proposed by extension of a method 
already used in the literature but for the linear models. It is tested the null hypothesis, at each sequential 
observation, that there is no change in the model against a change presence. The asymptotic distribution of 
the test statistic under the null hypothesis is given and its convergence in probability to infinity is proved 
when a change occurs. These results will allow to build an asymptotic critical region. Next, in order to 
decrease the type I error probability, a bootstrapped critical value is proposed and a modified test is studied 
in a similar way. 

Simulation results, using Monte-Carlo technique, for nonlinear models which have numerous applications, 
investigate the properties of the two statistic tests. 

Keywords: sequential detection, change-points, weighted CUSUM, bootstrap, size test, asymptotic behavior. 



1. Introduction 

Our aim is the construction of a test for detecting a change in a parametric nonlinear model F, = 
/(X,-;j8,-) + su i — 1,- •• ,n. The parameter fi will be first estimated by a parametric method and hypoth- 
esis test will be afterwards made by two nonparametric statistics. The test statistics we are going to consider 
are based on sequential empirical processes of parametrically estimated residuals. This problem appears in 
various fields, especially biology (for example: growth model or compartmental model), chemistry, industry 
(quality control), finance, ... 

Generally, there are two types of change-point problem: a posteriori and a /?n'on'(sequential). The a pos- 
teriori change-point problem arises when the data are completely known at the end of the experiment to 
process. For this model we begins by finding the change-points number; after that their locations and the 
regression parameters on each interval are estimated. In the case of a parametric a posteriori model with 



1 email: Gabriela. Ciuperca @ univ-lyonl.fr 



Preprint submitted to Elsevier 



February 28, 2013 



change-points we can give the following references: for a constant model with K change-points, a consis- 
tent estimator for K was proposed by Yao and Au (1988), using the least squares estimation method. If the 
errors are strongly mixing or long-range-dependent processes, always for a constant model, Lavielle and 
Moulines (2000) estimate the change-point number using a penalized least-squares approach. Bai (1999) 
proposes a test based on the likelihood for a linear model. Again, concerning the detection of a change in a 
linear model we can remind papers based on information criterion of Osorio and Galea (2005), Wu (2008) 
or still Nosek (2010). In a linear model, but with long memory errors, Belkhouja and Boutahar (2009) use 
several methods to detect the break number: three information criteria, a sequential parametric test and a 
procedure based on sum of squared residuals. A large class of time series with change-points are estimated 
by a semi-parametric framework, but for a known change number, by Bardet et al. (2012). For a parametric 
nonlinear model, with multiple change-points, a general criterion is proposed by Ciuperca (2011). For the 
detection of the change-point number by hypothesis test in a linear a posteriori model, we can remind the 
paper of Liu et al. (2008), where the empirical likelihood test was considered in the particular case to detect 
a single change in a linear model. Qu and Perron (2007) propose likelihood ratio type statistics to test the 
null hypothesis K changes, against the alternative hypothesis of K + 1 changes, always for a linear model. 
In the sequential change-point problem, which will be presented here, the detection is performed in real 
time. In a linear model, the most used technique is the CUSUM method. Horvath et al. (2004) propose 
two schemes to detect a change in a linear model, results which are improved, using the bootstrapping, by 
Huskova and Kirch (2012). The same method we find in Xia et al. (2009) for a generalized linear model. 
In the sequential change-point detection literature most researches consider the detection of a change in the 
random variable distribution (see e.g. Lai and Xing, 2010, or Mei, 2006). We can also recall several testing 
procedures proposed by Neumeyer and Van Keilegom (2009) for detecting the change-points in the error 
distribution of non-parametric regression models. 

In this paper, the real time change-point detection in a nonlinear model is studied. Generalizing Horvath et 
al. (2004) framework, a first test statistic is studied using the weighted CUSUM method, calculated after 
that the model parameters have been estimated be least squares method. Next, in order to decrease the type I 
error probability, following the idea introduced by Huskova and Kirch (2012) for the linear case, a modified 
test (of the first) by bootstrapping is considered. It is important to note that, the nonlinearity changes the 
results and the approach made by Horvath et al. (2004) and by Huskova and Kirch (2012) for the linear 
case. Above all, in a linear model, the least squares estimator of the parameters has an explicit expression, 
which facilitates the calculations and the results proofs. All results proofs are based on the explicit form of 
the estimator. In the nonlinear case, since the estimator expression is unknown and the regression function 
derivatives with respect to regression parameters depends on parameters and on regressors as well, imply 
that the theoretical results (and their proofs) are different. These problems are even more difficult to solve 
in a model where change-point occurs. Numerical algorithms will also change to calculate the critical value 
and test the break presence. On the other hand, in the paper of Huskova and Kirch (2012), the fact that the 
linear model contains intercept(see the Assumption influences in a important way the results. It 

is worth mentioning that we don't impose a discontinuity condition in the change-point for the model. By 
simulations, for two nonlinear models which have numerous practical applications, we obtain that the two 
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proposed tests have the empirical power equal to 1 and the empirical sizes widely smaller than the fixed 
theoretical size. However, the precision of the change-point estimator is the same by both methods. 
The paper is organized as follows. In Section 2, we introduce the model assumptions and some general 
notations. The construction of a statistical test and its asymptotic behavior are presented in Section 3. To 
decrease the type I error probability, Section 4 presents a modified test by bootstrapping. Next, simulation 
results illustrate the obtained theoretical results in Section 5. The proofs of the main results are given in 
Section 6, followed in Appendix by some Lemmas. 

2. Model and notations 

For coherence, we try to use the some notations as in Huskova and Kirch's paper, where the linear model 
was considered. 

Let us consider the following random parametric nonlinear model with independent observations 

Y{ = /(X,;/?,) + Si, i = 1, • • • , m, ■ ■ ■ ,m + T m . 

For the observation i, F,- denotes the response variable, X, is a px 1 random vector of regressors, the function 
/ : R p X @ — > R is known up to the parameters jS ; of dimension q x 1, fi l ■, e & c RQ, with a compact set. 
For the function / we make the classical suppositions for a nonlinear model: f(x; j8) is continuous in x and 
of class C 2 (0). For the function /(x;j8), we denote f(x;j8) = df(x;fi)/dfi and f(x;/3) = d 2 f(x;fi)/dfi 2 . We 
suppose that on the first m observations, no change in the parameter regression has occurred 

P i =0 > , for i = I,--- ,m, 

with /J the true value of the parameter on the observations 1 , • • • , m. The value of jS° is unknown. 
We test the null hypothesis, that for all the following observations, there is no change in the model 

Hq :j8, =j8°, m + 1 < i<m + T m , (1) 

against the hypothesis that there is a change to the m + k Q m + 1 observation 



The value of j3 m is also unknown. This problem has been addressed in the literature if function / is linear 
f(x;fi) = x'fi (see Horvath et al., 2004, Huskova and Kirch, 2012). Let be the sequential detector statistic, 
built as the weighted cumulative sum of the residuals, for < y < 1/2, k - 1, • • • , T m 




for m + 1 < i < m + 

for m + k® + 1 <i <m + T, 



(2) 



( 



T(m,k,y) = Hm+i<i<m+kSi/8(m,k,y) - 2 m +i<i<m+*l7i - /(X/,j8 m )]y >g(m,k,y) 



withrfm,*,y) S m 1 / 2 (l + £)( i y r , 



(3) 
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where fi m = arg ming YIJ = \ \Jj ~ /(X/; fi)] 2 is the least squares(LS) estimator of /? calculated on the observa- 
tions 1, • • • ,m. With this estimator we calculate the parametric residuals e, = Yj- f(Xf,fi m ), for i - 1, • • • ,k. 
Recall that the cumulative sum (CUSUM) of the residuals is YIi=m+\ s,-. Let be the q x ^-matrix B m = 

m 1 i f(X ! ;>8 )f (X,-; /?°) which is supposed non-regular for all m with probability one. Classic asymptotic 
results for a nonlinear regression (see also the relation (35)) imply jS m - B" 1 [m" 1 ££j f(X,;/? )£;] (1 + 
op(l)). The function g(m, k, y) of the relation (3), proposed by Horvath et al.(2004), is used as a boundary. 

Let us also consider the notations: A = ,E[f(X;;8 )], B = E[f(X;/3°)f (X;j6 )], A t = f (X ! ;;8 )B- 1 A, 
D = [a'B^a] , D A = A' A. Matrix B is supposed positive definite. All throughout the paper, vec- 
tors and matrices are written in bold face. 

The regression function, the random vector X, and the error Sj satisfy the following assumptions: 
(Al) (e,)i<i<n are i.i.d. and E[sj] = 0, Var[£j] - o 2 and £'[|e ! -| v ] < oo for some v > 2. 
(A2) f(x,/?) is bounded for all /? in a neighborhood of for all x € R p . 

(A3) For every i - 1, • • • , T m , the errors e, are independent of the random vectors X ; , for all j - 1, • • ■ , m + 
T 

(A4) (m + ly 1 Zm/CX.-;^ ) ^ W(X;j8°)], (/n + Z)" 1 Eft' f ( x -)8 ) ^ £[f(X;y8°)], 
(m + r 1 f(X,;;e )f (X ; ;yS ) ^ B for all / = 0, 1, • • • , T m . 

m—>co 

Assumptions (A2) and (A4) are made for the true parameter j8°, under null hypothesis Hq. For the pa- 
rameter 0^, under the alternative hypothesis, we request only the similar of (A4): 

(A5) (m + k^ + 1)' 1 Z^'f&a/t) ^ E[f{X^l)l{m + kl + l)- l Y^-^ +l ^i\0 m ) ^ E[f(X;/P m )l 
(m + k° m + IT 1 Zlf n+l f(X,;j8° )f (X,-;£° ) —> B, for all / = 0, 1 , • • • ,T m - k° m . 

The assumption that the nonlinear function / is continuous in x, of class C 2 in /? and also assumptions 
(A2) and (A4) are commonly used in nonlinear modeling and are necessary for the consistency and the 
asymptotic normality of the LS parameter estimator (see e.g. Seber and Wild, 2003). Furthermore, the two 
values yS° and m are interior points of the set 0. 

The error variance a 2 is unknown. To estimate it, on the historical observations i - 1, • • • , m, we consider 
an consistent estimator 

j m 

rt^—^j-f^firnrf- ( 4 ) 

q M 

For the errors, let us consider: e m+i t = (m + k)~ l YIJ=\ E j> £ 2 m+k - (m + k)~ l YIJ^ s 2 , and then, an another 
estimator for its variance besides of (4), built on the m + k first observations, is <x 2 k = (m + k)~ l 2ZT=\ k ( £ i ~ 

£m+k) 2 - 

Two cases are possible for the sample size, which will give different results, under the null hypothesis for 
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the test statistics: 

• T m = co, the open-end procedure; 

T 

• T m < oo, lim m ^oo T m - oo, with limbec = T > 0, with the possibility T - oo. In this case we have 
the closed-end procedure. 

Concerning the used norms, for a ^-vector v = (vi,--- ,v p ), let us denote by ||v||i = £y =1 |vy| its 
Z^-norm and ||v||2 = v 2 ) l/2 its L 2 -norm. For a matrix M = (a,,) i< ; <„ , we denote by ||At||i = 

13 1<7<9 

max y= i ... >9 (H^i I^O'D the subordinate norm to the vector norm ||.||i and by HAtlb = ^p{MMf) the sub- 
ordinate norm to ||.||2, with p(MM') the spectral radius of MM. 

All throughout the paper, C denotes a positive generic constant which may take different values in different 
formula or even in different parts of the same formula. All vector are column and \' denotes the transpose 
of v. We say that a random variable set (V n ) is bounded by a constant C with a probability close to 1 (or 
with a probability arbitrarily large): Ve > 0, 3n £ e N such that P[V n > C] < 1 - e. 

Now, a notation and a relation on the function g, used for the result proofs. Using the relation that for 
all x > we have < yf^ < 1 and that y e [0, 1 /2), we obtain that 

km- 1 ' 2 I k/m \ 1_r 

K m = sup — — = sup I ~ ~~. J €[0,1]. (5) 

i<k<°o g(m, k, y) i< k<0O \ 1 + k/m ) 

After from these general notations, in every section we shall give the notations used for each test. 

The proofs of all main results of Sections 3 and 4 are given in Section 6. To prove these results, necessary 
lemmas are stated and proved in Appendix (Section 7). 

3. Test by weighted CUSUM, without bootstrapping 

We are going first to build a test statistic based on the residuals = F, -/(X,;)8 m ) after the observation m 
by estimating the parameter /? on the historical data (T,-, X ; )i<,< m . The study of this statistic will be hampered 
by the fact that the estimator jS m does not have an explicit expression. 

The following Theorem is the generalization of the result obtained by Horvath et al.(2004) for the linear 
model, on the asymptotic distribution of the test statistic under the null hypothesis given by (1). We remark 
that, unlike to the linear case, the asymptotic distribution of the test statistic, under Hq, depends on the 
function fix; s ) and on the true parameter f3°. The value of T m , with respect to m, also influence the 
asymptotic distribution. 

Theorem 3.1 Let us consider the assumptions (A1)-(A4). Under the null hypothesis Hq specified by (l),for 
all real c >, we have 
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(i) IfT m - oo or (T m < oo and linim^oo T m /m - oo), then 



lim P 

m—>co 



— sup 

0~m l<k<co 



m+k 

z 

i=m+l 



Si 



/g(m,k,y) < c 



= P 



(\+t-D 2 t)\W(t)\ 
sup < 



< c 



0<t<-+y 



(6) 



(ii)IfT m < oo andX\m m ^oaT m jm — T < oo, then the left-hand side of (6) is equal to P sup 0<f< r (l+t ^ '^ w ^ 

Here {W{t),0 < t < ooj is a Wiener process (Brownian motion) i.e. a centered Gaussian process, with co- 
variance function Cov{W{s), W{t)) = min(s, t), s,t e [0, -^]for (i) and s,t € [0, 1+ ^ 2r ]/or (ii). 

In order to have a test statistic, thus, to build a critical region, it is necessary to study the behavior of 
the statistic in the left-hand side of (6) under the alternative hypothesis H\. By the following Theorem, 
this statistic converges in probability to infinity as m — > oo. For this, we suppose that the change-point k® m 
is not very far from the last observation of historical data. Obviously, this supposition poses no problem 
for practical applications, since if hypothesis Hq was not rejected until an observation k m of order m, we 
reconsider as historical data, all observations of 1 to k m . Another supposition is that, before and after the 
break, on average, the model is different, without imposing a discontinuity condition in the change-point. 

Theorem 3.2 Suppose that the assumptions (A1)-(A5) hold. Under the alternative hypothesis Hi specified 
by (2), ifk° m = 0(m) and E[f(X;/3 )] + E[f(X;/3° m )] hold also, then 



< c 



— sup 

Cm \<k<T„, 



m+k 
i=m+ 1 



/g(m, k, y) 



Considering the Theorems 3.1 and 3.2 we derive in the next corollary a test statistic for testing the lack of 
change against the break presence. 

Corollary 3.1 Consequence of these two theorems, following statistic can be used to test Hq against H\: 



Z y {m) - sup 

°"m \<k<T m 



m+k 

X 

i—m+\ 



Si 



lg(m, k,y). 



(7) 



The asymptotic critical region is \z y (m) > c a (y)}, where c a (y) is the (1 - a) quantile of the distribution 



of sup 0<K ^[r^(l + t - D z t)\W(t)\], if\xm m . 

~ ~ xP- 



T m /m = oo, and of mp 0<t< _r[r 7 (\ + t - Z) z t)\W(t)\], if 

\+D z T 

lim m ^oo T m /m - T € (0, oo). For some given a e (0, 1), this statistical test, consequence of Theorems 3.1 
and 3.2, has the asymptotic type I error probability (size) a and the asymptotic power 1. 

It is important to note that, in the linear case f(x;J3) - x'fi, the value of D depends only on Z?[X], E[XX'] 
but not on the values of /?°. For a nonlinear model, the critical values c a (y) depend on the regression function 
/, the distribution of random vector X and on parameter value /?° before the change-point. 
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Remark 1 In the linear case, the assumption that the model contains intercept, X = (\,X\, • • • ,X p ), fi = 
(bo,b\,--- ,b p ), imposed by Horvdth et al. (2004), is essential. If E[X\] - ••• - E[X p ] = 0, then it is 
necessary that the model has different intercepts before and after change-point. Without this supposition, 
the test statistic Z y (m) can not converge to infinity under H\. 

Therefore, we deduce from it that, the null hypothesis Hq is rejected in the change-point 

f m = { M 1 1 - k ~ Tm ' & ™ |r(m ' *' 7)1 - Ca(7) ) (8) 
\ oo, if & m l \T(m, k, y)\ < c a (y), for every 1 < k < T m . 

which we can consider as estimator for k®. 



4. Test by weighted CUSUM, with bootstrapping 



In order to improve the critical values of the test, thus, to decrease the type I error probability, we extend 
the method proposed by Huskova and Kirch (2012), which uses the bootstrapping to calculate the critical 
value, function of the observation position, after the observation m. 

Let us suppose that until the observation m + k, the hypothesis Hq has not been rejected yet. Thus, for 
/ = 1, • • • ,m + k we have that under Ho, using the relation (35) and the proof of the Lemma 7.1, the 
cumulative sum of the residuals defined by (3) can be approached 



T(m, I, y) = 



m+l . m m+l 

=m+\ V 7=1 / i=m+\ 



lg(m,l,y){l+o P {l)). 



m+k 



In order to realize the bootstrapping, let us consider the discrete uniform random variables tf m ^(i), for 
i - 1, • • • ,m + T m , such that P[K m ,k(i) - j] = l/(m + k), for j - 1, • • • ,m + k. We denote also by P* m k , 
E* m k , Var* m k the conditional probability, expectation, variance we respect to { < U m ,k(i), I < i < m + T m ], given 
(Yj, Xj)i<j< m+ t. The conditional expectation with the bootstrapped regressors is, for i - 1, • • • , m + T m , 

y'=i 

Keeping the same notations as in the linear model of Huskova and Kirch (2012), let us consider (see Section 
2, for the other notations), for k = 1, • • • ,T m , following notations 

DZ^rr, [l^m+l f(X/;j8°)l/<* + lZm+k-M fi^fi^Km+k + Km + k)~ l f(X r , fi°)h> m +k\, 



c l (m,k,l)-^ A - m] _^ i=m+ 
for 1 < I < T m . In the linear model, C\(m, k, I) depends only X, 



t(m,k,l,y)(si,--- ,s m+ i) 



lg(m, I, y), which is 



Zlm+i - \™ 1 X%i f (Xy^Vy) B-Jaim, k, I) 
an approach of the weighted CUSUM statistic T(m, I, y) given by (3), in order to facilitate the boot- 
strap. 
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s m ,k(j) = Yj - f(Xj;fi m+k ) are the residuals from the ordinary least squares method, with fi m+k = 
argrmn^+^/CX,;/*)]. 

£* mk (i) = £ m ,k(^m,k(i)) are the bootstrap errors. 

n2 



timator. 



the bootstrap variance es- 



• F* j(t (x) = Jl/o-^sup!^^ |f(m,*,/,7)(e^ /t (l)," - , e^(m + l))\ < x] a distribution function 
calculated using the bootstrap results. 

• For N > 1, let us consider F m>k = Y^Jq 1 aiF* mmax{{H)LQ) , for k = jL,--- , (j + \)L - 1 an other 
distribution function, proposed by Huskova and Kirch (2012) in order to accelerate the procedure. 
The positive constants a-,- are such that Z^q 1 or,- = 1. 

We note that in order to calculate the bootstrapped residuals £* mk (i), only the data (Y,-, X,-)i<;< m+ 7- m are 
bootstrapped, not the estimator jS m+yt of /3 calculated on not bootstrapped data. 

The (1 - a) quantile c m ^ a {y) at time m + k of the distribution F m ^ is obtained as the smallest real value such 
that 



Fm,k( c m,k;a(y)) > 1 - £*. 



(9) 



Contrary to the case of Corollary 3.1, for the weighted CUSUM statistic without bootstrapping, the critical 

values C m ,k;a 

(y) depend at the same time of m, and k besides a and y. 
Before to state the main results of this section, let us recall the Hajek-Renyi inequality (see Hajek and Renyi, 
1955) that is a generalization of the Kolmogorov inequality. 

Hdjek-Renyi inequality: if (G/t)i</t<n is a sequence of independent random variables with E[Gk] = 0, 
Var(Gt) < oo and {f>k)\<k<n is a non- decreasing sequence of positive numbers, then, for any e > and 
m < n, 



P[ max 

m<k<n 



b k 



A E[G 2 A 



j=m+l 



™ E[G 2 A 

L 

7=1 



b 2 m 



A particular case of this inequality is we consider b k = g(n, k, y), which is an increasing sequence in k, with 
the function g specified by relation (3). 

For the linear model (see Huskova and Kirch, 2012), to study the behavior of the distribution function F m , k , 
then the behavior of the statistic 1 /o^ k ) su Pi</<r m |f (m, k, I, y)(s* m • • ■ , s* m k (m + l))\, the Hajek-Renyi 
inequality alone was sufficient. In the nonlinear model, in the calculation of the bootstrapped residual s* m k , 

then of E m fr , the LS estimator fi m+k intervenes. Since fi m+k was not an explicit expression, we need a 
generalization of this inequality for random variable sequence of expectation converging uniformly to 0. 
First, we have the following general result. 
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Proposition 4.1 If {Z k>n ) l<k<n is a random variable such that E[Z k<n ] - fi kt „ — > 0, for n — > oo, uniformly in 
k, and for all e > and m < n, P[ma.x m < k < n \Z kt „ - Mk,n\ > e] — > 0, f/iere exists a natural number n e 
such that for n > n e , P[max m <£<„ \Z kt „\ > 2e] — > 0. 

As a consequence of the Proposition 4.1 and of the Hajek-Renyi inequality, a generalization of this 
last one can be established, for random variables with the expectation converging to 0. Let (Gj) x< - <n be a 

sequence of random variables such that E[G 2 -] < oo, for all j - 1,- • • ,n and E[b k l £* =1 Gj] - Hk, n —> 0, 
uniformly in k, for n — > oo, with the positive sequence (b k )\<k< n non-decreasing. Then, by the proof of 
Proposition 4.1, we have that, for any e > 0, there exists a natural number n e such that for n>n e 



max ■ 

\<k<n 



>2e 



< P 



I Tjj=i(Gj ~ E[Gj])\ 
max > e 

\<k<n b k 



(10) 



On the other hand, by the Hajek-Renyi inequality, we have for the random variable Gj - E[Gj], for any 

€ > 0, 

(11) 



\r j=l (Gj-E[Gj])\ 
max > e 

\<k<n b]<_ 



1 ^ Var[Gj] 



7=1 



But Var[G;] < E[G 2 ]. By the relations (10) and (11) it follows immediately that, for any sequence of 
random variables (G/) 1<7 - <(J such that E[G 2 j] < oo, for all j - 1, • • • ,n and E[bj} T/j = \ Gj] = Hk,n — * 0, 
uniformly in k, for n — > oo, with the positive sequence {bk)\<k< n non-decreasing and for any e > 0, then, 
there exists a natural number n e such that for n > n e , 



max > 26 

l</t<n Oyt 



7=1 



(12) 



Now, in order to study the residuals s m ^(i) = Yi - f(Xt; fi m+k ), calculated after observation m, we under- 
line, by a decomposition, the corresponding model error e,. Depending on the position of the observation "z" 
with respect to change-point m + k® t , where k^ is the change-point position under the alternative hypothesis 
H\ given by (2), and on the position of k with respect to , we have the decomposition for the residuals 

£ m ,k(i) = Ei+fiXtfiP)!^^ +f(X i ;$ n )l i>m+k o i -f(Xi;fi m+k )l^ m+k o [1^+1^ -f(Xi;fi m+k )l i>m+k o % >Jt o . 

Since fi m+k is the least squares estimator of A we have = ^^^^[^-/(X^^^+^X,;^ )]!^ 
+ K + L° m+ i * - /(X,;j8 m+ ,) + /(X, ;) S° )]!*>*<> • Then 6,-f (X,-; £ m4jfc ) = f(X i; fi m+k ) 

■f(X i; fi m+k ) - Zlf" fiX^fiXfX+k) - ^r = r + ?;f } f(Xi;fPjkx f ,fi m+k )l k>k0m . The statistic 



g(m, I, y)T(m, I, y)(s* m k (\), • • • , £* m k {m + I)) becomes 

2 SmjMnjj®)- -^f&fiflPtfmtWmMj)) 



i=m+ 1 



V 7=1 



BjciOn,*,/) = /i +/ 2 +n m , 



(13) 
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with / 



£ U,n, k «) and h 



- [k i:7=i t&fifftetl.tU)) B^CiOn, *, 0. The expression of <R m will 
be specified in Appendix (Section 7). We precise that the bootstrapped residuals are s m ,k(^m,k(i)) - 

With these elements, we can prove that the statistic f (m, k, I, y) is asymptotically determined by I\ and 
I2 under Hq and that each of them converges to a Wiener process. For these, we prove, by the following 
Proposition, that the term I2 can be also written asymptotically as a sum of s<u mk (j), by imposing a supple- 
mentary condition: 

(A6) for any £ > there exists M > such that P [max^,^ ||f(X;;j8°)|| 2 > m] < e. 

The proof of Proposition 4.2 is given in Section 6, where the nonlinearity intervenes decisively to prove that 
the sum of 1 to T m for the right-hand side of an expression like (12) converges uniformly in probability to 
zero. 

Proposition 4.2 Under the assumptions (Al )-(A4), (A6) we have for any e > 0, in probability, 

h-{-Umr; =l s<u mM ) 



SU P P m,k 

l<k<co 



max 

\<l<T m 



g(m, I, y) 



> e 



•0. 



Taking into account the proof of Theorem 3.1 concerning the asymptotic distribution of the weighted 
cumulative residuals sum T(m,k,y) calculated without bootstrapping, we show by the following results 
that the statistic f (m, k, I, y) bootstrapped has the same asymptotic behavior under Hq as T(m, k, y). Under 
hypothesis H\ , the term Km is asymptotically uniformly bounded and then, taking into account the relation 
(13), f (m, k, I, y) is uniformly bounded a.s. also (see in Appendix, sub-Section 7.2, the Lemmas 7.3 and 7.4). 



Proposition 4.3 Suppose that the assumptions (A1)-(A4), (A6) hold, 
a) Under the null hypothesis Hq, we have, for any ieR, 



sup 

\<k<T m 



m,k 



1 



0~m,k \<1<T, 



sup f (m, k, I, y)(s* mk (l), ■■■ , (s* mk (m + l))<x 



-P 



\Wi (j) - jOW 2 (l)\ 

SUp ; < X 



where {Wi(t);0 < t < 00} is a Wiener process, W 2 (l) is a standard normally distributed, independent of 
[Wi(t)}. 

b) If furthermore the assumption (A5) holds, under the alternative hypothesis H\, for any e > 0, there exists 
a constant M > such that, we have a.s. 



SU P P m,k 
l<k<T m 



J— sup \t(m, k, I, y){s* mk {\), ■■■ , s* mk {m + l))\ > M 



0~m,k 1<1<T, 



<e + Ml)- 
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As for the Theorem 3.1, under Hq, we can prove that the asymptotic distribution of sup^^ ^^+/7m)(f/^w+0)y 1 ^ 
is sup 0<f< i ( 1+? ~^y)iw(0l m case r m = oo or (T m < oo and lim m ^oo T m /m = oo). In the case T m < oo 
and limbec T m /m = T < oo, the asymptotic distribution is sup 0<f< t V+'-d >W(t)\ _ Combining Theorem 

i+z> 2 r 

3.1 with Proposition 4.3(a) under the null hypothesis, on the one hand, and Theorem 3.2 with Proposition 
4.3(b) under the alternative hypothesis, on the other hand, together with the distribution function definition 
F m ^, allow to define a critical value depending of each sequential observation k = 1, • • • , T m . Thus, we can 
define a new test statistic and study its asymptotic behavior under Hq and Hy . 

Theorem 4.1 Suppose that the assumptions (A1)-(A4), (A6) hold and that a e (0, 1), y e [0, 1/2). 
a) Under the null hypothesis Hq, as m — > oo, we have 



1 \T(m,k, 7 )\ 
— sup — > 1 

°"m \<k<T m C m fca\Y) 



a. 



b) If furthermore the assumption (A5) holds, under the alternative hypothesis H\, as m — > oo, we /zave 

1, 



1 \T(m,k,y)\ 
— sup — > 1 

°"m l</t<r m Cm,k;a\y) 



with T given by the vclatioti (3) and c m ^ a 

(y) fry f9J is critical value of the distribution function F m ^. 
Thus, we are going to use as test statistic of Hq, against H\ 
7(b) t , 1 \T(m,k,y)\ 

Zla(m) = — sup — , (14) 

0~m i<k<T m Cm,k;aKY) 

which will have the asymptotic critical region \z^ a (m) > l}. Then the statistic Z^) a {m) has asymptotic size 
a and asymptotic power one for all y e [0, 1 /2). As in Section 4, we consider the change-point estimator of 
kl is 



in 



(h) { inffl <k<T m , Pa^l > il 

1°°' ^^y) - ' for every l</:<r m . 

Then, hypothesis //o is rejected in t„\ Let us notice that, in comparison with the previous test, the value 
calculation c m ^ a (y) is little more laborious, in view of the fact that, the conditional distribution functions 
F* m k must be first calculated. 

5. Simulations 

In this section we report a simulation study designed to evaluate and compare the performance of the 
proposed test methods. For the two methods we consider two examples: growth model and compartmental 
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model for varied parameters, sample size or position of k m after m. For each test statistic, the algorithm 
steps are given to calculate the corresponding critical values. Afterward, details are given how to calculate 
empirical test size, empirical test power and to estimate the change-point location. 

All simulations were performed using the R language. The program codes can be requested from the author. 

5.1. Test by weighted CUSUM, without bootstrapping 

Firstly, following simulation steps are realized in order to calculate the critical values c a {y) in accordance 
with the Corollary 3.1: 



cess, respectively, taking into account the two possible cases (i) or (ii) concerning T m of Theorem 



3. On the basis of M replications of V y we calculate the critical values c a (y) such that P[V y > c a (y)] - a. 

A Brownian motion is generated using the BM function in R package^de). Once the critical values c a (y) 
are available, the change absence against the change of the model is tested using the statistic Z y (m), given 
by relation (7). In order to calculate the empirical test size, an without change-point model is considered 
and we count, the number of times, on the Monte-Carlo replications, when we obtain Z y (m) > c a {y). For the 
calculation of the empirical test power, the hypothesis Hi is considered true, that there exists a change-point. 
We fix T m = 500, k° m = 25 (or k° m = 2) and we vary the sample size m = 25, 100, 300, y = 0, 0.25, 0.45, 0.49, 
a - 0.025,0.05,0.10. For every combination, 1000 Monte-Carlo replications are realized. On the 1000 
replications, we computed the frequency among which the test statistic Z y {m) exceeds the critical value 
c a (y). In order to estimate the change-point location, we find the first point k in the interval 1, • • • ,T m such 
that (o- m ) _1 |ZSf+i H /g(m,k,y) exceeds critical value c a (y). 

For both models, in order to study the importance that f(x, jf?° ) is bounded or not, two regression parameters 
jS^ after the change-point are considered: one for which f(x,/f°) is bounded and another for which f(x,j8^) 
is not bounded. Even though the theoretical results are valid, we will study the precision of the change-point 
location estimator. 

5. 1. 1. Growth model 

Let us consider first the growth function f(x;fi) - b\ - exp(-&2x) which models many phenomena, 
with the parameters fi = {b\ , b-i) e 0, c 1 x I + compact and x e R. In this case the dimension of fi 
is 2 (q - 2) and it there is a single regressor (p - 1). We generate the response variable X ~ N(0, cr^) 
and the errors s ~ N(0, 0.5). The true values of regression parameters before the change-point are jS° = 
(0.5, 1) and after 0* m - (1, 2). By elementary calculations we obtain E[X exp(-&2^)] = -b2<r\ e\p{b^cr 2 x l2), 
E[X 2 e\p(-2b 2 X)] - cr|[l + Ab\] exp(2^cr|), then 



1. Calculate D= [a'B~ 1 a] 



2. Simulate M replications of the random variable V y - sup < ( <! ; S 2 
a Wiener process, or V y = sup < ; < t (l±fz^£« ; w i t h \w(t 




3.1. 



A = E[f(X,fi)] - 



1 

-bi<r\ exp(^2<x|/2) 
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Table 1: The (1 - or) quantiles (critical values) c a (y) of the random variable V r (specified in subsection 5.1, Step 2) calculated on 
50000 Monte-Carlo replications. Growth model. 



r l;a -> 


0.01 


0.025 


0.05 


0.10 


0.25 





2.7959 


2.5033 


2.2411 


1.9595 


1.5322 


0.15 


2.8581 


2.5690 


2.3058 


2.0313 


1.6146 


0.25 


2.9243 


2.6368 


2.3841 


2.1082 


1.7014 


0.35 


3.0220 


2.7536 


2.5044 


2.2414 


1.8462 


0.45 


3.2578 


3.0051 


2.7878 


2.5391 


2.1639 


0.49 


3.5214 


3.2668 


3.0473 


2.8040 


2.4133 



Table 2: Empirical sizes of test based on the statistic (7) for a growth model. Calculated for 1000 Monte-Carlo replications and 
T m = 500. 





a= 


0.025 




a = 


0.05 




a = 


0.10 




yl 


m=25 


m=100 


m=30() 


m=25 


m=100 


m=300 


m=25 


m=100 


m=30() 





0.0051 


0.0026 


0.0003 


0.0075 


0.0038 


0.0006 


0.0132 


0.0081 


0.0020 


0.25 


0.0058 


0.0025 


0.0023 


0.0084 


0.0043 


0.0043 


0.0154 


0.0075 


0.0079 


0.45 


0.0066 


0.0033 


0.0023 


0.0089 


0.0050 


0.0043 


0.0130 


0.089 


0.0079 


0.49 


0.0046 


0.0021 


0.0014 


0.0065 


0.0032 


0.0026 


0.0093 


0.0064 


0.0070 



B = E[f(X;fi)f (X;fi)] = 



1 ~ b 20- x exp(^cr|/2) 

-b 2 o-\ exp(^o-|/2) o-|[l + 4bj] Qy^{2b\o- 2 x ) 



Obviously D - 1 for any value of <x| and of the parameters &i,&2- This means that we obtain the same 
quantiles that in the paper of the Horvath et al.(2004). The empirical quantiles (critical values) c a (y) of the 
random variable V 7 are given in the Table 1. Based on these empirical quantiles, we are going to study 
the test size and its power for various values of m, y and a. We realize 1000 Monte-Carlo replication of the 
model and we take T m - 500. The empirical test sizes are presented in Table 2. We observe that the obtained 
values are smaller widely to the fixed a theoretical size . On the 1000 replications we found that empirical 
test power is 1, in any case. For the same parameters, we estimate now as follows the change-point location. 
For y = 0.49,0.25,y = and m = 25 or 100, after 10000 Monte-Carlo model replications in Table 3 are 
given the minimum, median, mean, third quartile and maximum of the change-point location estimations. 
For m = 300, the results are similar to those obtained for m = 100, thus we don't present them. We observe 
that the obtained change-point estimates are biased, and that considering either the median or the mean, 
there is a delay time in change-point detection. In the Table 4 we have the summarized results when the 
change-point is immediately later after m, for = 2. From these two Tables 3 and 4 we deduce that, with 
respect to y, when the change is in ]P m - 25, there is no difference concerning the location change-point 
precision. If the change is immediately (k^ - 2), the precision decreases when y decreases. 
In all tables, we indicated between "()" the obtained results when 0^ = (1, -0.5), case in which the function 
f(x; 0^) is not bounded for all x. The results are worse, even though the break in k Q m is largest. 
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Table 3: Estimation of the change-point location based on the statistic (7), for 10000 Monte-Carlo replications, T m = 500, k° m = 25, 
J3° = (0.5, 1), 0> m =(1,2) and between () for ff m = (1, -0.5). Growth model. 







m= 


25 




m = 


100 




r 


summary (f m ) I ; a — > 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


0.49 


min 


1(1) 


1(1) 


1(1) 


1(1) 


1(1) 


1(1) 




median(Q2) 


32 (41) 


32 (39) 


31 (37) 


31 (37) 


31 (35) 


31 (34) 




mean 


35 (46) 


34 (43) 


33 (40) 


34 (39) 


33 (38) 


32 (36) 




Q3 


39 (54) 


38(51) 


37 (48) 


37 (45) 


37 (43) 


36(41) 




max 


148 (279) 


148 (248) 


148 (232) 


112(144) 


111 (143) 


109 (132) 


0.25 


mm 


1(1) 


1(1) 


1(1) 


9(1) 


9(1) 


9(1) 




median( Q2 ) 


32 (39) 


31 (37) 


31 (35) 


32 (38) 


32 (37) 


31 (35) 




mean 


34 (43) 


33 (40) 


32 (37) 


34 (41) 


34 (39) 


33 (38) 




Q3 


39 (49) 


38 (46) 


36 (43) 


38 (46) 


37 (44) 


37 (42) 




max 


141 (245) 


141 (219) 


119(200) 


110(137) 


110(125) 


95 (125) 





min 


1(1) 


1(1) 


1(1) 


9(25) 


9(21) 


9 (20) 




median(Q2) 


33 (41) 


32 (39) 


31 (36) 


34 (43) 


33 (41) 


33 (39) 




mean 


35 (45) 


34 (42) 


33 (39) 


36 (45) 


35 (43) 


35 (41) 




Q3 


39 (52) 


38 (49) 


37 (45) 


41 (52) 


40 (49) 


39 (46) 




max 


121 (300) 


115 (284) 


115 (223) 


115(136) 


115 (132) 


115 (131) 



Table 4: Estimation of the change-point location based on the statistic (7), for 10000 Monte-Carlo replications, T m = 500, k a m = 2, 
s = (0.5, 1), j? m =(1,2) and between () for fP m = (1, -0.5). Growth model. 







m= 


25 




m = 


100 




y 


summary(f m ) l;a—> 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


0.49 


min 


1(1) 


1 (1) 


1(1) 


1(1) 


1 (1) 


1(1) 




median(Q2) 


6(6) 


6(6) 


5(5) 


6(6) 


6(6) 


5(5) 




mean 


8(10) 


8(9) 


7(8) 


8(9) 


7(8) 


7(7) 




Q3 


10(12) 


10(11) 


9(10) 


10(11) 


9(10) 


9(9) 




max 


109 (224) 


109 (218) 


79 (185) 


91 (81) 


91 (71) 


91 (67) 


0.25 


min 


1(1) 


1(1) 


1(1) 


1(2) 


1(1) 


1(1) 




median(Q2) 


7(8) 


6(7) 


6(6) 


7(10) 


7(9) 


7(8) 




mean 


9(H) 


8(10) 


7(9) 


10(12) 


9(H) 


9(10) 




Q3 


11(14) 


10 (12) 


10(11) 


13 (16) 


12(15) 


11(13) 




max 


74 (156) 


72 (133) 


72 (133) 


89 (94) 


87 (94) 


85 (94) 





min 


1(1) 


1(1) 


1(1) 


6(3) 


6(3) 


5(3) 




median(Q2) 


8(11) 


7(10) 


7(9) 


10(17) 


9(15) 


9(14) 




mean 


10 (14) 


9(13) 


9(11) 


12(19) 


12 (17) 


11(15) 




Q3 


13 (18) 


12(16) 


11(14) 


16 (25) 


15 (23) 


15 (20) 




max 


93 (210) 


88 (179) 


88 (175) 


91 (131) 


76 (124) 


73 (111) 
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5.1.2. Compartmental model 

Another very interesting nonlinear model, with numerous applications, is the compartmental model. 
Examples and references of important applications for these models are given in Seber and Wild(2003) (see 
also the references therein): it describes the movement of lead in the human body, the kinetics of drug 
movement when the drug is injected at an intramuscular site, etc... Consider two-compartment function 
hp(x) - b\ exp(-^ix) + b 2 exp(-b 2 x), P - (b\,b 2 ) £0£ K+. In this case q-2 and p - 1. 
As for the growth example, we consider a gaussian response variable X ~ N(0, o~ x ). For this model we 
have E[exp(-bX)] = exp(&V|/2), E[Xexp(-bX)] = -ba\ exp(6 2 cr|/2), E[X 2 exp(-2MQ] = cr|[l + 
4b 2 ] exp(2&V|). Then 



A = E[f(X,fl)] = 



(1 +^io-|)exp(^o-|/2) 
-(1 +b 2 o- 2 x )exp(b 2 2 o- 2 x /2) 



And with the notations B n - I + b 2 cr 2 (5 + Ab\) exp{2b\ cr|), B\ 2 = I + cr x [{b\ + b 2 ) 2 + b\b 2 {\ + {b\ + 
b 2 ) 2 )] exp((Z?i + b 2 ) 2 cr 2 /2), B 22 = l+ b 2 cr 2 (5 + 4b\)exp(2b\(r 2 x ), we have the matrix 



B = E[f(X;fi)f (X;fi)] - 



Bn B l2 
B12 B 22 



Contrary to the previous case, the value of £) depends on the variance cr 2 x of the random variable X and on 
the parameters of the growth function. Hence, for each value of 0° and of variance of X we need to calculate 
the quantiles. For the simulations, let us consider a\ - 1 and jS° = (1.2, 1). In this case D = 0.5741. 
The empirical quantiles c a (y) of the random variable V y , specified at the beginning of this subsection, are 
given in the Table 5. 

The simulations are carried out for historical data of size m = 25, 100 or 300 and T m - 500 observation 
after m. The empirical type I error probabilities are presented in the Table 6 calculated by 1000 Monte-Carlo 
replications. As for the growth example, the empirical power test is 1 for each value of 7, a, when = 25 
and/?" =(1,2). 

In Tables 7 and 8, the summarized results on the change-point estimations obtained on 10000 Monte-Carlo 
replications, varying m, y and theoretic test size a. Between "()" we give the results for fy m - (-0.5,2), 
when the function f(x; ) is not bounded for all value of x. 

We can make the following observations. As for the growth example, the results are less good in the case 
f (x; f$) not bounded: the method detects later the change and especially we have greater maximal values for 
the change-point estimation t m . In the two case, = 25 and = 2, the precision of f m decreases when y 
decreases. The change-point estimation is more precise than for the growth model. 

5.2. Test using the bootstrapping 

In this case, the calculation of the critical values c m ^ a {y) defined by (9) is more laborious. We go to 
see if the simulation results are better than by weighted CUSUM without bootstrapping, case in which it 
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Table 5: The (1 - a) quantiles (critical values) c a (y) of the random variable V y (specified in subsection 5.1, Step 2) calculated on 
50000 Monte-Carlo replications. Compartmental model, j8° = (1.2, , 1), cr\ = 1. 



y 4. ;a -> 


0.01 


0.025 


0.05 


0.10 


0.25 





6.2165 


5.5233 


4.9211 


4.2812 


3.2689 


0.15 


5.7627 


5.1279 


4.5862 


4.0014 


3.0854 


0.25 


5.4929 


4.9022 


4.3838 


3.8395 


2.9833 


0.35 


5.2355 


4.6960 


4.2092 


3.7024 


2.9142 


0.45 


5.0383 


4.5223 


4.0786 


3.5998 


2.9191 


0.49 


4.9682 


4.4702 


4.0555 


3.6032 


2.9945 



Table 6: Empirical sizes of test based on the statistic (7) for a compartmental model. Calculated for 1000 Monte-Carlo replications 
and T m = 500. 





a= 


0.025 




a = 


0.05 




a = 


0.10 




y 


m=25 


m=100 


m=30() 


m=25 


m=100 


m=3()0 


m=25 


m=100 


m=30() 





0.0003 








0.0005 








0.0007 








0.25 


0.0006 








0.0006 








0.0009 








0.45 


0.0010 








0.0013 








0.0014 


0.0002 





0.49 


0.0009 








0.0011 


0.0001 


0.0001 


0.0012 


0.0004 


0.0002 



Table 7: Estimation of the change-point location based on the statistic (7), for 10000 Monte-Carlo replications, T,„ = 500, k Q m = 25, 
/?" = (1.2, 1), fP m =(1,2) and between () for )8° = (-0.5, 2). Compartmental model. 







m= 


25 




m = 


100 




r 


summary(f m ) I ; a — > 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


0.49 


min 


1(1) 


1(1) 


1(1) 


26(1) 


5(1) 


1(1) 




median(Q2) 


30 (31) 


30 (31) 


29(31) 


29(31) 


29 (30) 


29 (30) 




mean 


30 (34) 


30 (33) 


30 (33) 


30 (32) 


30 (32) 


29 (32) 




Q3 


33 (37) 


33 (37) 


32 (36) 


32 (36) 


32 (35) 


31 (34) 




max 


63 (105) 


63 (102) 


60 (102) 


55 (105) 


53 (105) 


53 (102) 


0.25 


min 


1(1) 


1(1) 


1(1) 


26 (26) 


26 (26) 


26 (26) 




median(Q2) 


31 (32) 


30 (32) 


30 (31) 


30 (32) 


30 (32) 


30 (31) 




mean 


31 (35) 


31 (34) 


30 (34) 


31 (34) 


31 (34) 


30 (33) 




Q3 


34 (39) 


34 (38) 


33 (37) 


34 (38) 


34 (38) 


33 (37) 




max 


69(124) 


69 (124) 


59(115) 


61 (99) 


60 (99) 


52 (97) 





min 


3(1) 


3(1) 


3(1) 


26 (26) 


26 (26) 


26 (26) 




median(Q2) 


32 (34) 


31 (33) 


31 (33) 


32 (35) 


32 (34) 


31 (33) 




mean 


33 (37) 


32 (36) 


32 (35) 


33 (37) 


33 (36) 


32 (35) 




Q3 


36 (41) 


35 (40) 


34 (39) 


37 (42) 


36 (41) 


35 (40) 




max 


73 (143) 


67 (138) 


63 (114) 


69(120) 


69(120) 


66 (102) 
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Table 8: Estimation of the change-point location based on the statistic (7), for 10000 Monte-Carlo replications, T m = 500, k a m = 2, 
j6° = (1.2, 1), ff m =(1,2) and between () for ff m = (-0.5, 2). Compartmental model. 







m= 


25 




m = 


100 




y 


summary{f m ) I ; a — > 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 




min 


1(1) 


1(1) 


1(1) 


3(3) 


3(2) 


3(1) 


0.49 


median(Q2) 


5(6) 


4(5) 


4(5) 


5(5) 


4(5) 


4(5) 




mean 


5(7) 


5(7) 


5(6) 


5(7) 


5(6) 


5(6) 




Qi 


7(9) 


6(8) 


6(8) 


7(8) 


6(8) 


6(7) 




max 


29 (68) 


29 (61) 


29 (60) 


27 (80) 


27 (80) 


27 (49) 


0.25 


min 


1(1) 


1(1) 


Kl) 


3(3) 


3(3) 


3(3) 




median(Q2) 


5(7) 


5(6) 


5(6) 


6(7) 


6(7) 


5(7) 




mean 


6(9) 


6(8) 


6(8) 


7(9) 


6(9) 


6(9) 




Q3 


8(11) 


8(11) 


7(10) 


9(12) 


9(12) 


8(11) 




max 


37 (86) 


37 (86) 


35 (86) 


36 (98) 


35 (97) 


34 (90) 





min 


1(1) 


Kl) 


Kl) 


3(3) 


3(3) 


3(3) 




median(Q2) 


7(8) 


6(8) 


6(8) 


9(11) 


9(10) 


8(9) 




mean 


7(H) 


7(10) 


7(10) 


9(13) 


9(12) 


8(12) 




Qi 


10(14) 


9(14) 


9(13) 


13 (16) 


12(15) 


11(15) 




max 


35 (99) 


34 (91) 


34 (91) 


49(110) 


40 (91) 


40(91) 



deserves to make calculation effort. 

We now describe in detail the algorithm steps for calculate the critical values c m ^- a (y)- 
Step 1. We fix a, y, N, L, m, T m (see the notations given in Section 4 for N and L). 
Step 2. 

• We calculate J = T m /L; 

• For j - 0, 1, • • • , (/ - 1)L, the following random variable are generated 

Vj =" 4r SU P h 1 7)« /D, • • • , < j(m + 0)1 

m,j l ^ T " 

Step 3. For j - 0, 1, • • • , (J - \)L, we generate the random variables Wj which are mixtures of the random 
variables Vj generated to step 2. 

For each j — 1, • • • , (J - 1)L, we generate a multinomial distribution with parameters 1 (number of trials) 
and the probability vector pj - (1/ j, ■ ■ ■ ,1/ j). On the basis of this, thus, Wj - Vj for j - 0, 1, • • • 
with the probability 1 / j. 

Step 4. We repeat the steps 2 and 3 making M Monte-Carlo replications. At the end, we shall have M 
realizations for every random variable Wj, j = 0, 1, • • • , (J — \)L. 

Step 5. We calculate for every k - jL, jL + 1, • • • , (j + 1)L - 1 for j - 0, 1, • • • , / the random variables 
U k = Wj. 

Step 6. On the basis of M replications, for each k - 1, • • • , T m , we calculate the critical values c m ^ a (y) such 
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Table 9: Empirical sizes of test based on the statistic U k given in subsection 5.2, Step 5, for a compartmental model, for bootstrap 
critical values. Calculated for 1000 Monte-Carlo replications and T m = 500. 





a= 0.025 


a = 0.05 


a = 0.10 


r 


m =25 m=100 


m=25 m=100 


m =25 m=100 


0.25 
0.49 


0.0001 
0.0004 


0.0003 
0.0006 


0.0007 
0.0008 



that P[U k > c m ^ a (y)] = a. 

The change absence against the change of the model is tested using the statistic Z^(m) given by (14). 
In order to calculate the empirical test size, an without change-point model is considered and we count, the 
number of times, on the Monte Carlo replications, when we obtain Z^(m) > 1. Recall that the change-point 
estimation t$ is calculated using relation (15). 

Let us consider m = 25 and m - 100. For m - 300, the results are similar to those obtained for m - 100, 
thus we don't present them. In the case m = 100 we consider L - m/50 and in the case m = 25 we take 
L - m/10. For y we take only two values: 0.25 et 0.49. If &~ = 25, the empirical power test is 1 in all cases: 
for the two model type (growth or compartmental) and for the every parameters y and kP m . 
The same parameter settings are used as in the previous simulation study, in the subsection 5.1. 

5.2.1. Compartmental model 

The empirical test size based on the statistic Ut (of Step 5), calculated for 1000 Monte-Carlo replications 
and T m - 500, are given in the Table 9. By comparing the Tables 6 and 9, we deduce that the empirical test 
sizes are smaller by the bootstrapping method. 

The results concerning t„\ the estimation of k Q m , presented in the Tables 10 and 11, are almost the same 
for y - 0.49 and y = 0.25. Apart from y - 0.25 and - 25, the results for are not better than those 
obtained by the method without bootstrapping. 

5.2.2. Growth model 

Tables 2 and 12 indicate that the empirical test size obtained using the bootstrapped critical values are 
sharply lower than empirical test size without bootstrapping. Concerning the change-point estimation (Table 
13), for y = 0.49, m = 25 and &° = 25, the results for tj* } are better than by the weighted CUSUM method 
without bootstrapping. On the other hand, for y = 0.49, m = 100, the results are less good using the 
bootstrapped critical values. 

5.3. Conclusion on the simulations 

Two test statistics and their critical regions are, using weighted CUSUM method without and with 
bootstrapping for two nonlinear models. In both cases, the empirical sizes are widely smaller than the fixed 
theoretical size a. But the empirical sizes of test are without thinking smaller when the critical values are 
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Table 10: Estimation of the change-point location based on the statistic (14), for 10000 Monte-Carlo replications, T m = 500, 
k Q m = 25,0 = (1.2, 1), 01 = (1,2) and between () for/?° = (-0.5,2). Compartmental model. 







m= 


25 




m = 


100 




y 


summaryif^) J,; a -» 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


0.49 


min 


2(2) 


1(1) 


1(1) 


26 (26) 


26 (26) 


14 (26) 




median(Q2) 


32 (30) 


30 (28) 


26 (27) 


32 (35) 


31 (33) 


29 (32) 




mean 


32 (33) 


30 (38) 


27 (29) 


32 (37) 


31 (36) 


30 (34) 




Q3 


36 (36) 


33 (35) 


30 (33) 


35 (42) 


34 (40) 


32 (38) 




max 


90(118) 


78 (118) 


65 (92) 


72 (122) 


63 (122) 


54 (122) 


0.25 


min 


5(1) 


5(1) 


HI) 


26 (26) 


26 (26) 


26 (26) 




median(Q2) 


28 (30) 


26 (28) 


24 (26) 


32 (35) 


31 (34) 


30 (32) 




mean 


28 (32) 


27 (30) 


25 (28) 


32 (38) 


32 (36) 


31 (34) 




Q3 


32 (38) 


28 (34) 


28 (31) 


34 (41) 


34 40) 


34 (38) 




max 


66 (106) 


59 (103) 


57 (103) 


71 (134) 


66(118) 


57 (98) 



Table 11: Estimation of the change-point location based on the statistic (14), for 10000 Monte-Carlo replications, T m = 500, k° m = 2, 
= (1.2, \),0 m = (l,2)and between () for 01 = (-0.5,2). Compartmental model. 







m= 


25 




m = 


100 




y = 0.49 


summaryiT^) J,; a -» 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


0.49 


min 


3(1) 


2(1) 


2(1) 


3(3) 


3(3) 


3(3) 




median(Q2) 


6(8) 


6(6) 


5(5) 


6(7) 


5(7) 


5(6) 




mean 


8(11) 


7(9) 


6(7) 


7 (10) 


6(10) 


5(8) 




Q3 


10(13) 


9(11) 


8(9) 


9(14) 


7(12) 


7(10) 




max 


65 (76) 


64 (76) 


33 (63) 


43 (86) 


42 (75) 


27 (75) 


0.25 


min 


3(2) 


1(1) 


HI) 


3(3) 


3(3) 


3(3) 




median(Q2) 


6(7) 


5(6) 


4(5) 


7 (10) 


6(10) 


6(7) 




mean 


7(10) 


6(9) 


5(7) 


8(13) 


7(12) 


7(9) 




Q3 


9(13) 


8(12) 


7(9) 


11 (18) 


10(16) 


9(12) 




max 


38 (81) 


32(81) 


28 (62) 


37 (119) 


36 (74) 


32 (74) 



Table 12: Empirical sizes of test based on the statistic Ok given in subsection 5.2, Step 5, for a growth model, for bootstrap critical 
values. Calculated for 1000 Monte-Carlo replications and T m = 500. 





a= 0.025 


a = 0.05 


a = 0.10 


r 


m=25 m=100 


m=25 m=100 


m=25 m=100 


0.25 
0.49 


0.0005 
0.0002 


0.0012 
0.0003 


0.0027 0.0004 
0.0007 
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Table 13: Estimation of the change-point location based on the statistic (14), for 10000 Monte-Carlo replications, T m = 500, 
y = 0.49, J3° = (0.5, 1), fP m = (1, 2) and between () for 0> m = (1, -0.5). Growth model. 







m= 


25 




m = 


100 




k° 


summary (t^) I ; a — » 


0.025 


0.05 


0.10 


0.025 


0.05 


0.10 


25 


mm 


1(1) 


1(1) 


1(1) 


1(1) 


1(1) 


1 (1) 




median( Q2 ) 


27 (32) 


26 (30) 


25 (28) 


38 (53) 


36 (45) 


34 (42) 




mean 


28 (35) 


27 (33) 


26 (30) 


40 (57) 


39 (51) 


36 (44) 




Q3 


32 (39) 


32 (39) 


30 (36) 


48 (61) 


44 (60) 


42 (50) 




max 


107 (242) 


107(182) 


107 (175) 


146 (220) 


145 (184) 


140(156) 


2 


min 


1(1) 


1(1) 


(1)1 


1(2) 


1(1) 


1(1) 




median(Q2) 


5(6) 


5(5) 


5(5) 


8(11) 


8(10) 


7(9) 




mean 


7(8) 


7(8) 


6(7) 


11(19) 


10(15) 


9(12) 




Q3 


9(10) 


8(9) 


8(8) 


14 (26) 


13 (20) 


12 (16) 




max 


82(122) 


82(122) 


80 (107) 


122 (192) 


104(150) 


101 (100) 



calculated by bootstrapping. The power test is equal to 1 for any value of m, y, k® m , or theoretic test size a. 
The both test statistics (7) and (14) detect the change produced in the model. 

The parameter y does not modify the type I error probability. Concerning the change-point estimation 
precision, it does not improve in a significant way by the bootstrapping method or when the number m of 
historical data increases. This precision can be influenced by y value when the test statistic (7), without 
bootstrapping, is used. It is worth mentioning that the obtained estimations of &~ by the both methods are 
slightly biased, the delay time is of order - +6 observations, either for m = 25 or for m - 100 observations. 
Finally, if f(x,/?) is not bounded, the both test statistics detect the change-points, but the estimator bias of 
increases, if the change is 2 observations after m or 25 observations after m. 

6. Proofs of the Theorems and Propositions 

Here we present the proofs of the results stated in Sections 3 and 4. 

Proof of Theorem 3.1 

The proof follows the structure of the Theorem 2.1 proved by Horvafh et al. (2004) for the linear case. 
(i) Using Lemma 7.1 and Lemma 7.2 we have 



m+k ' m+k ]c m ^ 

sup V Si/g(m,k,y) = sup V s t Y^m 



/g(m,k,y)(l+o P (l)) 



- (T sup [W hm (k) - D-W 2 (m)]/g(m, k, y)(l + o P (l)). (16) 

l</t<oo m 

with Wi >m and W2 two independent Wiener processes on [0, 00). We obtain in a similar way as in the linear 



20 



case (Theorem 2.1 of Horvath et al., 2004) 



W hm (k) - ±DW 2 , m (m)\ £ \wm - iDW 2 {m)\ 
sup — — sup — — , 

i</t<oo g(m,k,y) iik<oa g(m,k,y) 

where { Wi(0), { W^COI are two independent Wiener processes on [0, oo). For all K > 0, by the continuity of 
{Wi(t) - DtW 2 (l)/(t/(l + f)Y) on [0, K] we have 



\Wm-±DW 2 (m)\ £ 
max — - — = max 

\<k<mK g(m,k,j) l<k<mK h | k \l k \ 

\ mj\ m+k j 



M a . s . \WM-DtW 2 (\)\ 

sup 



i.s. 

The relations (5.9) and (5.10) of Horvath et al.(2004) hold, then, for all 6 > 0, 



(17) 



lim lim sup P 

K >0O — >on 



sup |Wi(-) - -£W 2 (1)|/(1 + -X-^-rY - DW 2 {\) 

mK<k<oo mm m m + k 



> 6 



= 0, 



lim P 



\Wi(t) - DtW 2 (\)\ 
sup DW 2 (\) 



> 6 



= 0, 



thus 



\W hm (k) - %DW 2 , m (m)\ L \Wi(t) - DtW 2 {\)\ 
sup — - — — > sup . 

i</t<oo g(m,k,y) m ^°°o<?<oo (l+0(TT7) r 



(18) 



Let us consider the random processes Z(t) = Wi{t)-DtW 2 {\) and U(t) = (1 +D 2 t)W (j^), with { W(t), < 
t < oo} a Wiener process. Their variances are Var[Z(t)] = t+D 2 t 2 = ?(l+£> 2 0, Var[U(t)] = (\+D 2 t) 2 t/(l + 
D 2 t) = t(l + D 2 t). For t x < t 2 , Cov{Z{h),Z{t 2 )) = E[Z(ti)Z(t 2 )] + D 2 ht 2 = h + ^hh = ti(l + D 2 t 2 ) and 
since t/(l + D 2 t) is increasing in t, Cov(U(h), U(t 2 )) - (1 + £> 2 ?i)(l + D 2 t 2 )hl(l + !D 2 h) = h(l + D 2 t 2 ). 

Thus, their variances and covariances coincide, we have Z(t) - U(t), for < t < oo. Let us make the change 
of variable f/(l + D 2 t) - y, hence 



mjt) - tW 2 (\)\ L lw , (l+y-D'yy 
SU P —r, — t: t w = SU P l w tv)l • 



<te<» (1 + 0(i^P 



(19) 



()<y< 



~1? 



By the asymptotic properties of a nonlinear regression, we have that the variance error estimator cr 2 m is 
strongly converging to a 2 , \& m - <r\ - op(l). The assertion (i) follows by the last relation together the 
relations (16), (17), (18)-(19). 
(ii) The proof is similar of (i). We give its outline: 



m+k 



sup V E t /g(m,k,y) = cr sup [W hm (k)-D—W 2 (m)]/g(m,k,y)(l+o P (l)) 

l<k<T mi ^ +l 



l<k<T m 
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\Wi(t) - DtW 2 (l)\ l .„„ „(1 +y - &y) y 



Proof of Theorem 3.2 

We choose this particular k: k m = k^+m. We will prove that for this k m we have lim m ^oo H^f+i s,- /g(m, k m , y) 
oo. Let us consider the partial sum of the residuals after the first m observations 



m+k,„ 



m+k m m+k m 



m+k m 



(=m+l (=m+l i=m+l i=m+k° m +l 

Similar as for the Theorem 3.1 we have, for the first two terms of the right-hand side of (20), 



(20) 



lg{m~k m ,y) = P {\) 



m+k m 

£ [si + fiXr^-fiXiM] 

i=m+ 1 

and for the last term of the right-hand side of (20) 

m+k m m+k m 

£ r/(X I -;^)-/(X i ;^°)]= J] [/(X«;/0-£[/(X;j8°)]] 



(21) 



i=m+k% + 1 



- ^ [/(X,;/? ) - E[f(X;0)]] + (k m - k°J (E[f(X;fi°J] - E[f(X; y8 )]) . 

i=m+k^+l 

Since £[/(X; /?° )] + E[f(X; s )}, which implies, for the third term of the right-hand side of (22) that 
(km ~ k Q J\E[f(X;0J] - E[f(X;0)]\ Cm 



m l / 2 g(m,k m ,y) 



m (\ + Isl\( ~ k »i /m )y 



> C > 0, 



(22) 



(23) 



where C is a constant not depending of m. For the last relation, we have used that for x > 1 we have 
\ < yf^ < 1, then ( j^j^ y e (2^, 1) and (1 + x)" 1 > 1. On the other hand, using assumption (A5) 



m+km 



m+k m 



m+k" 



£ [f(Xr,fi° m ) - E[f(X;Pl )]] - ^ [/(X ; ;/0 - W(X;/?° )]] - £ [/(X ; ;jS° ) - E[f(X;tf n )]] 

i=m+k°+\ i=l i=l 

is of order 6>p(m + & m ) 1/2 + 6>p(m + k^) l/2 - Op{m + & m ) 1/2 . Moreover m~ l {\ + k m /m)~\m + k m ) l/2 —> 0, 
asm^oo and ( km J m ) y e (2~ r , 1). Thus, for the first term of the right-hand side of (20) we have 

1 iKfu j til 

m+k m 

m- 1 ' 2 £ [f(X i; /&)-E[f(X;fi m )]yg(m,k m ,y) = op(\). (24) 

i=m+k m + 1 
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Similarly, for the second term of the right-hand side of (22) 

m+k m 

m- l/2 £ I/(X i ;^°)-£|/(X;^ )]]/g(iii,^ l ,7) = op(l). (25) 

i=m+k% + 1 

Taking into account the relations (22)-(25) we can get, for the third term of the right-hand side of (20) 

m+k m 



lim inf m 

m—>co 



■1/2 



J] [/(X ; ;jS°)-/(X i;) S )] 

i=m-\rk l L+l 



lg(m,k m ,y) > 0. 



(26) 



The relations (20), (21), (26) imply that we found one k m such that lim^oom 1/2 Z^f+i &i lg{m,k m ,y) > 
Yl^lx e t lg(m,k m ,y) = co. Then lim„ su Pl < fc<00 s\ lg(m,k,j) = °°- The the- 



0. Thus lim m ^oo 
orem follows. 



Proof of Proposition 4.1 

It is clear that \Z k n \ — n kn < \Z^ n —fj. k>n \ ^ 

max k \Z k 

,n l^k,n\- Then P\rs\ax k \Z kn n kn 
e] = P[\Z k J > e + \nk, n \] > P[max k (\Z kn \) > e + |yujt, n |] > Ptmax^dZ^I) > 2e]. For the last inequality we 
have used: for all e > there exists a natural number n £ such that for all n > n e we have \nk, n \ ^ € - • 

Proof of Proposition 4.2 

We denote by e 2 = -i/mH™] £f U,„k(j) anc ^ we rem i n d the notation Da = A f A. Without loss of generality, we 
take / < k, the other cases are similar. Consider now the following random variable, for i - m + 1 , • • • ,m + l, 



Gi,k = — 



1,k(f)- 



£ f (Xj;/?)su mM fCX,;^ ) - -J- J £ % 
Consequently, -72 + <?2 = Z™t«+i ^i>*- The conditional expectation of -I2 + ^2 is 

E* km [-h +e 2 ] = -T f ? (X y ;/3°) V SiB-Jdim, k,l)--Y V s t 

m ' m + k m ^— ' m + k ^ 



= £m+k \ D A l 



/III 



1=1 



7=1 



i=\ 



m 



yyi _|_^ ffl }11 

V f (X,;/3 )^- V f(X,/3°) - ™ V f (X,;/3°)I V f ( X,/3<>) 



-I 



On the other hand, by assumption (Al) for all e > 0, there exists Mi > such that P[m l ^ 2 \E m+k \ > Mi] < e. 
Thus, taking also into account the assumption (A4) for f(Xj;/3°), we get 



J k,m 



-12 + &2 



g(m, I, y) 



o P (m + l)s m +k o P {\ + l/m)m 1/2 s m+k 
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Using the relation (5), the last relation is op(\)m l/2 s m+ j c - op(\)Op(\) - op(l), for all /, k - 1, • • • , T„ 
Consequently 



'k,m 



Zm+l f 
i=m+l "i.* 



g(m, I, y) 



= o P {\). 



for all I, k - 1, • • • , T m . Then, we are in the conditions to apply the inequality (12) for the random variable 
G it/ t and the sequence (bi = g(m, I, y)). Hence, for any e > 0, there exists a natural number m e such that for 

m > m £ , 



k,m 



sup — — — > 2e 

i<l<T m g(m,l,y) 



i Is, E*. [G 2 ,] 

1 y k,m L i,k J 

~ el £i g 2 (m,l,y)' 



(27) 



By elementary algebra, using the fact that for j * f, E* km [su mM - et/ m ,,(/)] = KJ- e ^^ E l^ e ^m^ = 

(m + ky 2 (r::^af, yiew 



+ 



1 Z7=i [D- A l t (X;;/?°)f(X;;j8 ) - 
(? m+ * - (s m+k ) 2 ) m- 2 27 =1 (D^f (X y ;/? )f(X i; /? ) - l)' 



(28) 



By the relation (30) of Huskova and Kirch (2012), we get, for a constant C\ > 0: g(m, I, y) > Ci(m 1/2 ~>'Z^l;< m + 
m~ l/2 lli >m ). Thus, for / > m we have g~ 2 (m,l,y) < mC~ 2 r 2 < C~ 2 mT x — > 0, as m — > oo and for 1 < / < m, 
g~ 2 (m, Z,y) < C~ 2 m" 1+2 ''/" 2 >' < C~ 2 m _1+2y -> 0, as m -> oo. Under the assumptions (A4) and (A6) we have 
the following inequalities with a probability close to 1 

r i2 
On the other hand, by the Cauchy-Schwarz inequality we readily have with a probability 1 



1 Tm \ m i t J 2 

Z('< x ^)'< x '^'- 1 ) ■ 



l=m+l L 7= 1 



1 



g 2 (m, I, y) 



< i m 4 2 



/=H1+1 



— 

^ f (Xfi/Pv&nfiPwi 1 - l 

7=1 



4, 1/2 



1/2 



< C(r m - in) V 1 <\C(T m - m)m 2 V j 



1/2 



i i 



1/2 
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were used that rn x £™ 1 f (X 7 ;j6°), m~ l YT j=x f (Xj;fi°)f(Xj;/3 ), 7^ f (X y ;j8°) are converging by as- 
sumption (A4). Hence 



ZKf(x^)f(x^)-i) 



/=i L;'=i 



1 



g 2 (m, Z, y) 



= P (\). 



(29) 



For the second term of the right-hand side of the relation (28) we have: f (X y ; j8°)f(X,; j6°)f (X;;/? )^,;/? ) 
= trace({(Xr, ^fiXf^f (X y ;y8 )f(X /;j S )) - f (X /;j 8 )f(X i;j S )f (XfifPjt&nfP). Consequently, since 
for I < m, g~ 2 (m, Z, y) < Cm' 1 ^, we have with a probability close to 1 that nT 2 £J» i £ J =1 {f (X y ; y8°)f (X,; jS )^- 1 - 
l}V 2 ("«.'.r) is less than or equal to 2m" 1 2™j f (X / ;^°)B m f(X / ; ) 8 )D- 2 + 1 g' 2 (m,l,y) < Cm~ 1+2 i 'rn x 

■YZxihxv^JiX^D- 2 + 1] < Cm-'^m- 1 Z"i[ll0I 2 f'(X/;^)|l!l|B m || 2 + 1] - Cm 1+2 ^ -» 0, for 
m — > oo. For the last relation we used the assumption (A4). 

We have in the other hand m~ 2 £ J" m+1 Z£i (f (X y ; j8°)f (X,;/? ^ 1 - 1 J g~ 2 (m, Z, y) is less than or equal to 
2m- 1 ZL+I g- 2 (m,/, 7 )m- 1 Zf =l [off (Xf, /3 )f(X f , 0>)f (X ; ;/? )f(X /; /? ) + 1 
- 2m" 1 Ej" m+ i g~ 2 (m, Z,y)[D~ 2 f (X,;/? )B m f(X/;j6 ) + 1] and by the Cauchy-Schwarz inequality 
< 2m" 1 [ll^ 2 f(X /; ^)|| 2 ||B m || 2 + I] 2 } 1 '' rV ^T)) 1 



1 1/2 



1/2 



< Cm 1 < (r m - m) ^ g 4 (m, Z, y) \ < Cm 1 — > 0, for m — > oo, 



(30) 



l=m+\ 



uniformly in fc. Since £ m+/ t — » and e 2 ■ - {s m+ k) 2 - d 2 h — > cr 2 , and using the results (28), (29), (30), 

m—>co m ' lc m— »oo 

we obtain by (27) that su Pl < fc<00 P* m [su Pl < teTm G«| /g(m, Z, y) > e] 0. ■ 

Proof of Proposition 4.3 

Using the Proposition 4.2 and the Lemmas 7.3, 7.4, the proof is similar to that of the Lemma 5 of Huskova 
and Kirch (2012). ■ 



Proof of Theorem 4.1 

Using the Proposition 4.3, the Theorems 3.1, 3.2 and Lemma 7.5, the proof is similar to that of the Theorem 
1 of Horvath and Kirch (2012). ■ 
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7. Appendix 

In this section useful Lemmas to prove the main results of Sections 3 and 4 are given. We recall the 
notations: A = E[f(X;0)], B = E[f(X;0)f(X;0)], A, = f'(X l -;/3°)B- 1 A. 

7.7. Lemmas for Section 3 

Lemma 7.1 Suppose that assumptions (Al )-(A4) hold. Under the hypothesis Hq we have, as m — > oo, 



sup 

1<&<oo 



m+4 / m+£ 



=m+l \i=m+l i=l 



lg(m,k,j) = Ml)- 



Proof of Lemma 7.1 

Under the hypothesis H , Y^+x ^ = ^tLx *~^tL\ [/(X«;y8 ffl )-/(X ( ;/i )]. Then, by a Taylor expansion 
of /(X,;j8) in a neighborhood of 

m+k ' m+k , m N , m m+k , m m+k 

y s~ y £; --VA; £i =_YA iei -y [f&rfj-f& r ,?)] = -Y. y f(x i;j s m ), 

i=m+l \i=m+l i=l I (=1 i=m+l i=l i=m+\ 

(31) 

with fi m = + 0(fi m - 0), 6 e [0, 1] ? . We know that the LS estimator /? m of parameter in a linear model 
is V"^- cons i stent ( see Seber and Wild, 2003) fi m - = Op(m~ 1/2 ). On the other hand, by the triangle 
inequality 



m+k 



m+k 



2 f(X ; ;/3J - kE[f(X;0)] < £ (f(X ; ;j8 m ) - E[f(X;0)]) + £ (f(X,-;^ m ) - E[f(X;0)]) 

(32) 



i=m+ 1 



1 <=1 



1 i=l 



Generally, for any n > m and /? in a m 1 ^-neighborhood of by assumption (A4) for f(X ! ;yS°), using the 
law of iterated logarithm we have that for all e > 0, there exists a M 2 > such that P[n~ 1/2 £" =1 ||f(X,-; j8°) - 
A||i > M2] < 6. Together with assumption (A2), it holds that, for all jS in a mr 1/2 -neighborhood of 

n n n 

£ [f(X,;j8) - E[f(X;0)]] < ^JfCXi^-fCX,-;^ )] + ^ [f(X,;/3V £[f(X;/3 )]] 



i=i 



!=1 



(=1 



- -0\\ l O P (n) + P {n 112 ) = P (n ■ m~ 1/2 ) + P {n 112 ) uniformly in jS. Thus, the right-hand side of (32) 
becomes (m + k)m~ 112 + (m + k) l/2 + m l/2 + m l/2 - (m + k)m~ l/2 + (m + k) l/2 + 2m 1/2 . Hence, for the last 
term of (31) we have 



sup • 

l<k<ca 



0m -fi°Y Z7=m+i (f(X«;/5 m ) - E[f(X;0)])\ 



g(m, k, y) 



= P (m~ l/2 ) sup 



m l/I + {m + k)m 1/2 + {m + k) 
k~V( k V 



a/2 



mi/ 2 (l + i)(^) 
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1 + (i + ±) + (i + i) 1/2 

= c sup V t m) iN \ m > < C sup 

k ' l<A:<oo 



2 + * 



mi/2 (! + i) (_A_y - - ^ ml/2 (l + y ^ _^ 

Since, for k < m, ^ > ^ and for x > we have (1 + x)" 1 < 1, we can write 

3 



2 + i 



sup 

\<k<m 



>" , (' + i)(i- 1 ir) T "-" J (4) r 

For all x > 1, we have that (^) y < 2? and (1 + x)" 1 < 2" 1 . Then 

1 /2? 



= 3 • 2^ 1 m- 1/2+ >' = o(\). 



2+± 



sup 

m<k<oo 



m 



!/2 \ 2 



- + 2* =o(l) 



Hence, (31) becomes 



V e,- - V £i : - - Y Atsi = - Y Aje,- + *# M - j8°)'E[f (X; j8°)](l + Ml))- 

i=m+l \i=m+\ i=l I i=l 



On the other hand, fi m is the least squares estimator of calculated for i - 1, • • • , m, 

= Zr=i f(X I ;j8 m )[F i - /(X,-; &,)] = 2" j f (Xjtfjfe - 08 m - (X,-; &,)] 

- Zr=i f(X/;y8 ) £i + Zr=i 'i&M(fim - - XZi f(X f ;j8°)f (X,-; j8°)08 m - j8°) 

-i/208 m kxf,fi m )f(Xi-,^)(fi m -fy 

Using the assumptions (Al), (A2) and the Cauchy-Schwarz inequality, we obtain 



= 



m m 

^ f(X ; ;y3 ) £; - J] f(X i; j8°)f (Xi-0)(fi m -j8°) 



!=1 i=l 

Then, by relation (34) below 



(l+Ml)). 



{ \ m \ { \ m 

fi m -0 = B m i - y f(x !;> 8 o ) £! - (i + mi)) = b- 1 - y f(x i; ^ o ) £i - 



(l+Ml)X 



again too 



(33) 



(34) 



(35) 



v m i t m 

fc08 OT -/PyElkXf,/?)] = - y t&nfihei B- x £[f(X;)8 )](l + Ml)) - - Y A ;£; (l + Ml)) 



i=l 



27 



and we replace next in (33). To complete the proof, we must prove that for (34) that sup^^ k/(mg(m, k, y))op(l) = 
o P (l). Using (A1)-(A4) and the fact that fi m -fi° = P (wr 111 ) we deduce that 

sup —, - y < ^m-ll V f(X,-;j8 m )e,-||i - Ml), 

with given by (5). Similarly 

ll^ IB -^)'^Z^i{f(X I ^)f r (X I ^)-£[f(X;^ )f'(X;^ )]}|| 1 

sup — - — = £ m Mi) = Mi)- 

i</t<oo fc, y) 

Using (5), with the Cauchy-Schwarz inequality for matrix, we have 

\Wm -A )' IZi f(X«;yS m )f(X^°)08 m -0>y\\ 2 

su p ? — 

i<i<oo g(m,k,y) 

, m 

< sup atji- V i(Xi-,P m )f(Xr,flP)\h ■ UP m -Ah ■ P (\\fi m - j8°||i) = Mi). 

i<k<oo m j-f 



Lemma 7.2 Suppose that assumptions (A1)-(A3) hold. Under the hypothesis Hq, there exists two indepen- 
dent Wiener processes [W\, m (i), < t < oo} and {W2, m (t), < t < oo} such that, for m — > oo, 



sup 

\<k<ca 



' m+k k m ( 

Vi=m+1 (=1 / x 



^-(rW^m) 
! m 



/g(m,k,y) = Ml)- 



Proof of Lemma 7.2 

The random variables {Y*T= m +i £ *' 1 ^ £ < °°l and {H™ i A,e,} are independent. It is obvious that, since X,- 
is independent of e ; , we have E[A,-Sj] = 0, Var[A,e,] = E[A?]i?[e?]. On the other hand, E[ej] = o 2 and 
E[Aj] - A'B _1 A. By an argument similar to the one used in Horvath et al.(2004), Lemma 5.3., we obtain 
sup lst<00 \LT=Li £ i ~ o-W hm (k)\ lk l 'y = P {\) and YZi A i £ i - <r£>W 2<m (m) = o P (m l ' v ), as m -» oo, y > 2. 
The rest of proof is similar to that of the Lemme 5.3. of Horvath et al.(2004). ■ 



7.2. Lemmas for Section 4 

We recall that (see the decomposition of gf given in Section 4): g(m, I, y)t(m, I, y){s* m fc (l), • • ■ , £* m k (m + 
0) = h + h + f^m- More precisely, 7? m have the decomposition: 7? m = £y=3 with 



m+l 



h = ^ 1 k<k%^ m , k (i)<m+kl [/(X-W mji (i);^°) - /(X-W^CO^m+Jt)] > 



!=«!+ 1 
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m+l 



i'=m+ 1 
m+l 



!=m+ 1 
( 



\ t 

~ Zi ' ( X i ; ^ 0)1 -t</t» ^^..O^m+C [/( X -Wm,tO);^°) - /( X -W^O)^m+fc)] 



7=1 



1 m ? 



7=1 



-2f(X 7 ,^)l^o 



7=1 



B^Ci^A:,/), 
B^Ci(m,A:,/), 
B^CiCm.A:,/). 



Under the hypothesis Hq, I4 - I5 - Ij - 1% - 0. Let us consider now 

. m+k , m+k 

7 3 = / 3 - __ 2][/(X 7 ;^ ) - /(X 7 ;j8 m+fc )], 7 6 = / 6 + _— ^[/(X;;/? ) - f(X r ,fi m+k )]. 
m + k m + k 

7=1 7=1 

Lemma 7.3 Under the assumptions (A1)-(A4), for all e > 0, we have 
|/ 3 1 



m,k 



max 



i<l<T m g(m, I, y) 



> £ 



SU P P *m,k 
l<k<ca 



max 



|/ 6 I 



> e 



in probability, uniformly in k, as m — > 00, 
0, m probability, as m — > 00, 



(36) 
(37) 



i</<r m g(m, Z,y) 
whether under Hq or H\. 

Proof of Lemma 7.3 Let us consider the random variable, for i = m + 1, • • • ,m + I, 

~ m+k 

Z k , m {i) = f&U^fiP) - f^m^m+k) - — T X [/( X 7-;/*°) - f( X pfim+k)l (38) 

m + k *r-f 

7=1 

Then£; jZ fc)m (0] - (m+^c)" 1 Zp k [f(X r ,fi )-f(X r ,fi m+k )]-(m+kT l Zp k [f( x f,fi )-f( x j;L+kV = 0. 
Hence, since 7 3 - Z^n'+i Zk,m(i), we have - 0. 

For /~6 let us consider I < k, for two other cases the arguments are like. Since !(, is scalar, using (A4), from 
an equality to an other we apply the trace operator, the conditional expectation E* mk [-I^] is equal to 



m 



j m+k j m 

— ^[/(x,;/? ) - f(Xr,fi m+k )] - J] f(x y;j e°) 

1=1 V 7=1 



( m+l 



^m+k^m+k^A 



2 f(X^°) 



\i=m+\ 
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r\— 1 m+k j rn-rt m / 1 m \ 

m + k*— * m + l*— i m / — 1 m ^— 1 

i=l L (=1 i=l J V i=l / 

= [ (m + /)A f - mA ; ]D^A(l + o P (\)) Y[/(X ; ;/?°) - f(X r ,fi m+k )] 

m + k 

, m+k 

= 1(1 + o P (l))—- Yt/CX,;/? ) - f(XiJ m+k )l 
m + k 1 

Hence, E* mk [I 6 ] = o P {l)^- k JgyU&tP)- f&*P«»ki\ = l(m+k)-^ 2 o P (l) = foT 1/2 op(l). On the other 
hand, the conditional variance of Z,k,m(i) is 

1 m+k 

Var* mk [ZkAm = E*[f(Xu m , k (iy,/3°J - f&it^; fi m+k )l 2 = — k Z^/* ) - mrX+k)] 2 

i=l 



Wm+k-0>\\l 

m + k 



m+k 



Z H f ( X -/*°) 

(=1 



(\ + o P (\)) = C(m + ky\\ + o P {\)), 



which is op(l) uniformly in k. We apply the Hajek-Renyi inequality for Zk,m(i)' for all e > 0, 



1 



max 



i</<r m g(m, Z,y) 



m+l 



Z z ^ ( ° - e -^Z 



l £ ^K^Wl . l c 



g 2 (m,l,y) e 2 m 



i=m+\ 

and the relation (36) follows. 

For !(,, we can write, h-~ ZyLi f ( X 7';J e °)^,«i(j)B^ 1 ci(m, Zc, Z). Then, since 



C_ y 1 _ C 

+ ^ i~t g 2 ( m > i> y) f2 ( m + 



l/m 



g(m,l,y) (i + Z / m )(_J_) r 



op(l) = 



l/m 
1 + Z/m 



l-y 



op(l) and 



l/m 
1 + Z/m 



is bounded by the relation (5), 



combined with sup^^ sup l£[< ^ Tm ||ci(m, k, OlbM ^ C an d since the random trial of bootstrap are indepen- 
dently, then Zk,mU) are also independently, we have Var* mk [I 6 ] = m~ l Var* m k [Zk,mU)\c\(m, k, l)B^ +k a(m, Zc, Z) 
op(\). Hence, by the Bienayme-Tchebychev inequality, Proposition 4.1 and inequality (12), the relation (37) 
follows. ■ 

Let be (the expressions of UJsJi, h are given before the Lemma 7.3): 



7=1 j=m+k%+l 

/ 7=^7+l*>*o— t J] [/( X 7^ )-/(X,;j8 m+fc )], 7 8 =/ 8+ l^_ ^ [/(X ; ;j8°)-/(X 7 ^ m+ ,)]. 
m m + k ' m m + k *—i 



j=m+k^+\ 
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Lemma 7.4 Under the assumptions (A1)-(A5), if the hypothesis Hi is true, we have that for all e > there 
exists M > such that we have in probability 



SU P P m,k 

l<k<co 



max 



> M 



\<l<T m g(m, I, y) 
Proof of Lemma 7.4 

For 7 4 . Let be consider the following random variable 



<e + o P (l), for je {4,5,7,8}. 



(39) 



j Ml+C 

(=i 



It is obvious that E* mk [Z m ,k(i)] = 0. Since I 4 = Z m ,k(i), we have that E* mk [I 4 ] = 0. For the 

conditional variance of ZLmjfS) we have, using a quadratic Taylor expansion and the triangular inequality 

V<*[Z»*(0] = E* [l^ (l)<m+ ,o [/(X^ (0 ;j8°) - /(X^ (0 ;j8 m+fc )]] ~ ^ [f(X r ,^)-f(Xj;fi m+k )] 



7=1 



=— y 

7=1 



(fim+k-^y,, 



( m + k m )\\fi m+k -/3% 
m + k 



IB 



m + k 



>0ii2 



£ f(X r ,fi°)f(X r ,fi m+k )(fi m+k -fi°) 



7=1 



+ 



l^-/H I^ 

4(m + &) 



^ ||f(X y ;^)||2. 



7=1 



-In v m+ '» 



By the Bienayme-Tchebychev inequality we have that (m + f (X;;/J°)f(Xj;j3 m+jt )||2 < (m + 

k)~ l Z™^ llf (X y ;^ )||2l|f(X y ;)8 m+ ,)|| 2 < ((m+k)- 1 f (X y ;^ )f(X y ; ) 8 )) 1 / 2 ((m+^)- 1 HfCX,;^)!^) 1 / 2 . 

Since is compact, together the assumptions (A2), (A4), we obtain that Var* mk [Z, m ,k(i)] ^ C. As in the 
linear case, using YJ^i g~ 2 ( m > Uy) ^ C, we obtain the inequality (39) for I4. 

Fori-]. We can prove in a similar way as in the Lemma 7.3, that E* mk [g~ l (m, Thebychevl,y)Ij] - op(l). 
Similar arguments as for !(, of the Lemma 7.3, unlike that Var*[Ij] = Op(\) uniformly in k and probability 
1, we obtain the relation (39) for j = 7, by the Bienayme-Tchebychev inequality, Proposition 4.1, and in- 
equality (12). 

For I5 and Ig . We consider the random variable defined by 

^ m+k 

Zm,k(0 - fl <« m ,*(i)>m+*° [f&UnjcUhfim) ~ f( Xr U m M'^m+k)] ~ £ [ ^ X 7^m) _ f( X pfim+k)l 

i=m+k°+l 
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and the results are proved by a similar way using assumption (A5) on the place of (A4). 



We recall the notations <t« 2 = (m - qT l Z^K^CO - m~ l Z%i f (Xy^^O^B^fCX^ )] 2 and 
^mk - ( m + ^) 1 YIi=\i E i ~ £m+k) 2 - We will prove that, under hypothesis Hq, cr 2 mk and <r , are two 
uniformly consistent estimators for the variance o 2 of the errors e 2 . Under hypothesis H\, this two statistics 
are significantly different. 

Lemma 7.5 Suppose that the assumptions (A1)-(A4) hold, 
a) Under the hypothesis Hq, we have, in probability, 



SU P P m,k 

l<k<T m 



0~m,k 



*.(*) 



cr 



m,k 



> e 



0. 



b) If furthermore the assumption (A5) holds, under the hypothesis H\, for all e > 0, there exists a constant 
M > such that 



sup P*„ 



l<k<T m 



m,k 



0~ m,k 



-(*) 

cr , 

m,k 



> M 



< e + o P {\). 



Proof of Lemma 7.5 

We have the decomposition, for each i of 1 to m: s* mk (i) - (m _1 £™ =1 f (X^^e^O^B^fCX,-;/? ) = 
Y? j=l Jj(m,k,i), where: ^(m,*,*) - £ttmk(i) , J 2 (m,k,i) = -(m l £J =1 f (Xy^^^jB"^^,-;^ ), 



h(m,k,i) = l^l^!^^^ 



/(X^ (0 ;^°) - f(X<u mM ;fi m+k ) - — ^[/(X,;/* ) - /(X,;jS m+fc )] 

7=1 



J 4 (m,k,i) = l w ol %1 (i )<ra+t o [/(X-w m ,,(/);^)-/(Xt/ m , (/) ;^ m+i )]-ll fc> ^— — J] [f(X-j;flP)-f(X.j;fi m +k)\, 

7=1 
^ m+i: 

/ 5 (m, *, - l^o l^ m , (0>m+(t o [/(X %a(0 ;/3° ) - /((X^ (0 ;/* m ^))]-l te4 o — — - J] [/(X 7 -;^)-/(X 7 ;/? m+it )], 



/ 6 (m, fc, - 



1 m < 



A:" [/(X«W m , t 0>^°) - /((X^.tO^^m+zt))] 



7=1 



B^fCX,;/? ) 



m+k 

+— r X [/(x ; ;>8 ) - f(x r ,fi m+k )i 

m + k 



7=1 
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Jj(m, k, i) = 



in 

V J =1 



= 1 



B^X,-;/? ) 



J s (m,k,i) - 



-J]f (X f ,j3P)l k>k o 



7=1 



+ ^>c^n Z [/(X;;j8°)-/(x y;) 8 m+ ,)]. 



;'=m+A:!i+l 



Then 



j m 8 

o- w , 2 V V Jj(m,k,i) 



(=1 L7'=l 



(40) 



Under //o, we have J^,J=,,J-],J% - 0. 

Following results hold under the two hypotheses Hq and H\ . For /i we have that for any e > 



SU P P m,* 

1<£ 



1 1 



m- qd 2 , . . 

1 m,k i=l 



^ j\{m, k, i) - 1 



> € 



(41) 



For J2, we have E* mk [J2(m, k, i)] = -s m +km~ 1 f (X y ;y8 )B- 1 f(X,;)S ). Then, using the independence of 
e,- and of X,-, assumption (A4), we obtain the convergence in probability, uniformly in k, as m — > 00, 



^ m . m t 1 m 

£ « *[- Z ^ *, 0] - -*m + * - J" f (X;-; JS°)B^ - V f(X,-; j8°) 
' m 4—* m 4— ' m 4-^ 

(=1 \ .7=1 '=1 



0- ABA = 0. 



We have also the approximation of E* m k \m- x YZ\ 4( m > k > 0] by B^E^nT 2 YT j=x f (X y ;jS°) {(XfifiP)^^ 
+2m- 2 Zf±j f (X y ;j8°)f (X 7 ; )S ) £%a0 - )£%a(/) ] = B;, 1 [m^B^m+A:)- 1 2™+* £ 2 +2m -2 ^ . ^ f^/jO) 
•f (X,v ; y8°)((m + A:)" 1 IT-f £ a ) 2 ]. But (m + ky l Y%-\ e a -A and (m + /i)' 1 IT-f e 2 , A cr 2 . Hence, we 

J " -1 " -1 m— >oo " -1 m— >oo 

have uniformly in k 



(42) 



(=1 
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On the other hand, we can write Jj,{m, k, i) = Z.k,m(i), with Z.k,m(i) defined by the relation (38). By the proof 
of the Lemma 7.3, since Var* m k [Xk,m(i)] - (m + fc) _1 B(l + op(l)) and the Bienayme-Tchebychev inequality, 
we have in probability 



SU P P m,k 

k>\ 



— — ^ jj (m, k, i) > e 



m - q 



0. 



(43) 



For J X J 2 we have wT x YZ\ k < MOn, k, i) = - [m' 1 Y? j= \ f (X;;/jV%aO)) B m 2* 1 f(X 7 ;j8 ) e %,,, 
and then, as for the calculations from above for Ji, we have E* mk \m~ x YIi=\ Ji(m,k,i)J2(m,k,i)] = op(l), 
uniformly in k. 

For JiJj, we have, using the assumptions (A2)-(A4), E* mk [m~ l £™j Ji(m,k,i)Ji,(m,k,i)] = m~ l H^Km + 
fc) _1 X£? sjlfQLfifih-f&fifi^)]. -w(m+^)- 1 X$ ej[f(Xj;flP)-f(Xj;P m +J\} = o P {\)-o P {\)o P {\) 

op(l). We show similar for the other cases that E* Jm~ l Y™_, J\{m, k, i)Ji(m, k, /)] — > 0, / = 3,4, • • • , 8. 

m,K i-i >oo 

The conditional expectation E* mk [m~ l YIi=\ h{tn, k, i)J^(m, k, i)] is equal to 



^ m . . m+k 

(=1 7=1 



) 



+ £ m+k 



B; 1 -2f(X,^) — ^[/(X,,^)-/^;^)], 

y=i J v ,-=i ; 7=i 



which converges to 0, from m — > oo, uniformly in fc, since e, is independent of X,-, e m+ ^ — > 0, and all terms 
in X, are bounded. 

a) Under hypothesis H Q . For J(,{m, k, i) we have 



E* mk [J 6 {m,k,i)] = -(0 m+k -P°y 



m + & ^— 1 

a=l 



> f(X«;jS u ) 



i Zf(X7^) 

7=1 



B^ffX,;/* ). 



Then, under assumption (A4), E* m k \rn x Yl? = \ him, k, i)] = -08 m+yt -/?°)'A • A' • B _1 A(1 + o P (l)). Similarly, 
with C a constant, EVJ^V, Ml < lbS m+ * ■ HAA'H* • IIB" 1 ^ • f(X, ;> 8 )f (X ; ;jS°)(l + Ml)), thus 

K,kl m ~ l 2™i ^("».*.'0] < l^-^lli-IIAA'lll.llB^lll.HBIb-Cl+Ml)) and^m" 1 £f =1 / 6 4 (m,M)] - 
CII&h* -jS°|£(l + Ml))- Hence, under H , Var^lm- 1 ^ J 2 6 (m,k,i)] = o P (l). By the Bienayme- 
Tchebychev inequality, we have in probability: 



&U P P m,k 

k>\ 



— ^ Jl(m, k, i) > e 



m - q 



!=1 



0. 
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The relations (40), (41) and since all other conditional expectations for & m ' k expression are negligible, imply 
the assertion (a). 

b) Under H\ . For b e {4, 5, 6, 7, 8} we will prove that for all e > there exists a M > such that 



sup P 

k>\ 



m,k 



III 



— — ^ J\(m, k, i) > 



M 



i=l 



< € + Op(\). 



(44) 



In view of the previous calculus for J$ we have that the relation (44) holds for b = 6. 

For J${m, k, i) we have J^(rn, k, i) - Z,k,m(i) and by the proof of the Lemma 7.4 and the Bienayme-Tchebychev 
inequality, we have that the relation (44) holds for b = 4. 

For Jj, its conditional expectation E* mk [Jj(m,k,i)] is, by Taylor expansions and using assumptions (A2), 
(A4), 



m + k 



m+kl 

a=l 



7=1 



f (X ! ;^°)(l + o P (l)). 



Similarly E^m- 1 ^ J 2 (m,k,i)] = C|^° - fi m+lc \\ 2 2 (l + Ml)), ^[m" 1 1 /*(m, *, /)] = C|b8° - 
An+fcH^l + which imply the relation (44) for By Bienayme-Tchebychev inequality, we obtain 

(44) for J$ and Jg using assumption (A5) on the place of (A4). 

Now we consider the product of the terms of different suffix. The products of 73 with J4, J5, Jj, Jg, of 
J4 with /5,/6,/8 and of J5 with Jj are 0. For J1J4 we have E* Am Yli = \ Ji(m,k,i)J4(m,k,i)] = 



(m + ky 1 T!J^"'[sj - £ m+k ][f(Xj;fl°) - f(X.fifi m+k )l But, for all e > there exists M > such 



that 



(m + /t)- 1 EjJ^ |/(X 7 ;^°) - f(Xj-,fi m+k )\ > M < e, from which, together the fact - 0, one may 

deduce that 

p 

E* m i-[ m ~ l Zfci Ji(m,k, i)J4(m,k,i)] — > 0, uniformly in fc. By similar arguments we prove the uniformly 

m,/c 1 m— >oo 

convergence to in probability, for all other combinations of J 2 and J 4, ■ ■ ■ ,Jg. The not insignificant 
terms are J%, J 2 , J 2 , J 2 . We consider now J4J7, the other cases are similar. Taking into account the 
fact that ElJf&u^ifif&u^P)] is equal to E^[f& Um ^ o ;0)}E*[f&<u m Mj);P)] for i * J and 
to E* [f 2 (X<u mj!{i y,fi)] for i = j, we have that E* m ArrT l YZi Mm, k, i)J 7 (m, k, /)] - 

— V [/(X,;/? ) - /(X y ;)8 m+)t )] 2 - V f (X,;/? ) B; 1 - y f(X i;> 8 ) (1 + 0P (1)), 
1 + k) *r-f I m -r— r w > 1 



m(m + A:) 



i=l 



7=1 



which converges to in probability, uniformly in ^. 

Hence, in conclusion, taking into account the relations (40), (42), (43) and (44), 



-2(*) 
m,k 



1 



m - q 



^(/j (m, fc, + /^(m, fc, /) + /g(m, fc, /) + 7y(m, i) + /g(m, fc, /)) 
(=1 



(1 + 0P (1)) 
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>-V/J(ffl,t,0(i+Mi)) 

1=1 

and the assertion follows by (41). 
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